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(1) Real Party in Interest 

A statement identifying by name the real party in interest is contained in the brief. 

(2) Related Appeals and Interferences 

The Examiner is not aware of any related appeals, interferences, or judicial 
proceedings which will directly affect or be directly affected by or have a bearing on the 
Board's decision in the pending appeal. 

(3) Status of Claims 

The statement of the status of claims contained in the brief is correct. 

(4) Status of Amendments After Final 

The appellant's statement of the status of amendments after final rejection 
contained in the brief is correct. 

(5) Summary of Claimed Subject Matter 

The summary of claimed subject matter contained in the brief is correct. 

(6) Grounds of Rejection to be Reviewed on Appeal 

The appellant's statement of the grounds of rejection to be reviewed on appeal is 
correct. 

(7) Claims Appendix 

The copy of the appealed claims contained in the Appendix to the brief is correct. 

(8) Evidence Relied Upon 

2002/0040375 Simon et al 4-2001 

2003/01 63786 Shields et al 1 0-2002 

6208360 Doietal 3-1998 
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(9) Grounds of Rejection 

The following ground(s) of rejection are applicable to the appealed claims: 
Claim Rejections - 35 USC § 101 

1. 35 U.S.C. 101 reads as follows: 

Whoever invents or discovers any new and useful process, machine, manufacture, or composition of 
matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the 
conditions and requirements of this title. 

2. Claims 23-26, 28-31 and 35-40 rejected under 35 U.S.C. 101 because the 
claimed invention is directed to non-statutory subject matter. 

Re claims 23, 26 and 31 : the claims are towards a seemingly patentable computer 
readable medium. However, these claims are deemed to be non-statutory since the 
mere claiming of a computer readable medium with instructions is not a patentable 
process without a computer being used to realize the functionality of the instructions on 
the computer readable medium. It is suggested that the applicant modify the claims to 
ensure that the claim language reflects that the computer readable medium is encoded 
with instructions that are executed by a computer in order to perform some steps or 
process. The dependent claims are also rejected. 

Claim Rejections - 35 USC § 102 

3. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 1 02 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public 
use or on sale in this country, more than one year prior to the date of application for patent in the United 
States. 
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4. Claims 3-6, 8-11,1 5-1 7, 23-26, 28-31 , 35-37, 43 and 46 are rejected under 35 
U.S.C. 102(b) as being anticipated by Simon et al (US Pub No 2002/0040375). 
Re claim 3: Simon et al discloses a method of organizing digital images on a page, 
comprising: 

Identifying a set of digital images (i.e. in figure 5, the step 120 allows the images 
used to be arranged on a page format to be selected manually or automatically; see fig. 
5; paragraphs [0049]-[0055]); 

identifying a pre-determined print size for each of the digital images in the set 
(i.e. shown in the illustrated prior art image in figure 2 is a template with the 
identification of a pre-determined print size for each of the digital images to be used on 
the template. Since this template is used to identify pre-determined print sizes of 
pictures, this performs the above feature. Also, when the system employs the method 
of automatically placing pictures in the layout, the system chooses, or identifies, a 
certain pre-determined size on the overall layout to be printed. With this pre-determined 
space, the images chosen have a pre-determined print size and this size is taken into 
account when placing the images on the overall layout. The print size of the images is 
used to assist the system in arranging the images in a more aesthetic manner. The 
images in the system have a pre-determined print size since these images where taken 
by different sources (i.e. digital camera or scanner) and these different sources define 
the print size for the images. Therefore, the pre-determined print size feature is 
performed. The system may then normalize the pictures to take away the left over 
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white space on the overall layout when these images are placed on the layout; see figs. 
1-3; paragraphs [0046]-[0055]); 

defining a packing area (i.e. when the format of a page is selected in step 110, 
this is analogous to defining a packing area. In this step, the height and width of the 
page is specified in order to define where and within which format the images are to be 
placed on the page; see fig. 5 and 6; paragraph [0049] and [0050]); 

identifying a largest of the predetermined print sizes (i.e. shown in the illustrated 
prior art image in figure 2 is a template with the identification of a pre-determined print 
size for each of the digital images to be used on the template. Since this template is 
utilized to identify pre-determined print sizes of the pictures, this performs the above 
feature. When the system employs the method of automatically placing pictures in the 
layout, the system chooses, or identifies a certain pre-determined size on the overall 
layout to place the picture on the layout to be printed. In the system, an image may be 
identified as being much larger or smaller than other images. The image that is much 
larger than the others is considered as the image with the largest pre-determined print 
size. This image is identified when it is chosen to be placed on the layout and the 
smallest dimension of the image may be normalized in order to create a spatial balance 
between itself and other images used; see figs. 1-3; paragraphs [0046]-[0055]); 

if it will fit in the packing area, packing a digital image from the set having the 
identified largest pre-determined print size in a first orientation in the packing area in a 
first trial pack (i.e. the amount of images placed on a page is determined automatically 
based on the layout and the amount of space that allows the images to achieve a 
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spatial balance on the specified layout. If the image does not assist in creating a spatial 
balance or there is not enough room to fit an image on the page, the current image is 
not chosen automatically for placement. The system determines the size of an area to 
place an image and judges whether that area should be packed with the image that will 
not overlap other images. This image has a certain print size that needs to be taken 
into account before placing the image in a certain place with other images. With this 
process being performed, this is an example of identifying an image with a pre- 
determined print size to place on the layout in order for the image to not overlap other 
images. Once the image is placed, it can be determined that this image is the largest 
image placed on the layout compared to other images. Next, the normalizing process 
occurs to make sure that the image that is the largest is normalized in some manner to 
create a better spatial balance in the layout, and as a result, the amount of white space 
on the layout is minimized. Moreover, this process occurs until there can not be any 
more images laid out on the trial layout and a score is calculated for the trial layout; see 
fig. 7 and 11-14; paragraphs [0049]-[0055]); and 

if it will fit in the packing area, packing the digital image from the set having the 
identified largest pre-determined print size in a second orientation in the packing area in 
a second trial pack (i.e. during the process that is illustrated in figure 7, the above 
process that occurs to the first or prior trial layout occurs to the new or the second trial 
layout. The new trial layout shows another orientation of the same images being used 
due to the specifications given by the aspect ratio of the page format. Like the above 
explanation, this same system again determines the size of an area to place an image. 
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The system judges whether an area should be packed with the image for another trial 
layout based on if the user has expressed dissatisfaction or the system compares 
layouts with different scores. In this second layout, again the images with a pre- 
determined print size are placed on the layout and these images are placed in a manner 
that will provide ample space for the image to be placed and not overlap any other 
images in the layout. A normalizing process may occur to assist in improving the visual 
aspect of the layout. This process again occurs until there can not be any more images 
laid out on the trail layout and a score is calculated for the trial layout; see fig. 7; 
paragraphs [0049]-[0055] and [0057]-[0063]). 

Re claim 4: Simon et al discloses the method of claim 3, 

wherein packing the digital image in the first orientation includes, if the digital 
image from the set having the identified largest pre-determined print size will fit in the 
first orientation, packing as many digital images from the set having the identified 
largest pre-determined print size as possible in the packing area in the first trial pack 
(i.e. the invention finds images of certain sizes that may fill the trial layout in an efficient 
manner. This may be a large or small sized image. The images chosen to fill the trial 
layout shown in figures 8-10 are either the same size or a different size and the feature 
of packing as many digital images of a certain size as possible in a certain orientation in 
a trial pack is performed. The images in the system that are used have a pre- 
determined size to be printed on the layout. The images are placed on the layout with a 
pre-determined size and these sizes can be the largest or the smallest to be placed and 
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printed on the overall layout. The invention can pack large images, compared to other 
images, on a layout as long as the images do not overlap. These images can be 
packed until no other images can be placed on the layout anymore. Then, the process 
of normalization may occur to make the overall layout have a better aesthetic look; see 
figs. 5-10; paragraphs [0049]-[0057] and [0059]); and 

wherein packing the digital image in the second orientation includes, if the digital 
image from the set having the identified largest pre-determined print size will fit in the 
second orientation, packing as many digital images from the set having the identified 
pre-determined print size as possible in the second orientation in the second trial pack 
(i.e. when comparing which trial layout is the most efficient in figure 7, the same process 
that occurred to the first or prior trial layout also occurs to the new or second trial layout. 
The second trial layout is another orientation of the same images and therefore, the 
above feature is also performed. Since the same process for the first orientation occurs 
for the second orientation, the feature of identifying a set of images to place within a 
pre-determined print space is also performed. Like in the first orientation, the images in 
the system that are used have a pre-determined size to be printed on the layout. The 
images are placed on the layout with a pre-determined size and these sizes can be the 
largest or the smallest to be placed and printed on the overall layout. The invention can 
pack large images, compared to other images, on a layout as long as the images do not 
overlap. These images can be packed until no other images can be placed on the 
layout anymore. Then, the process of normalization may occur to make the overall 
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layout have a better aesthetic lo; see fig. 5-10; paragraphs [0049]-[0055] and [0059]- 
[0063]). 

Re claim 5: Simon et al the method, wherein: 

identifying a largest of the pre-determined print sizes (i.e. shown in the illustrated 
prior art image in figure 2 is a template with the identification of a pre-determined print 
size for each of the digital images to be used on the template. Since this template is 
used to identify pre-determined print sizes of the pictures to be used, this performs the 
above feature. When the system employs the method of automatically placing pictures 
in the layout, the system chooses, or identifies, a certain image with pre-determined 
size to be placed on the overall layout to be printed. The image can be the largest or 
the smallest image out of the images chosen. This performs the feature of identifying 
an image with the largest pre-determined print size to be placed in the layout and 
placing the image or images on the page. The system may then normalize the pictures 
to take away the left over white space on the overall layout; see figs. 1-3; paragraphs 
[0046]-[0055]), comprises 

identifying, from a set of digital images, a largest pre-determined print size that 
will fit in the packing area (i.e. although identifying the largest pre-determined print size 
is not specifically stated, it is performed by the device. With normalization, the largest 
sized image is isotropically scaled so that the shortest dimension of the image is 
equivalent to the other images used in the page format. The image with a certain size, 
which is identified, is the image that will create a spatial balance in the page format. 
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This image can be the largest or the smallest size. The image will still be normalized to 
fit the image together with the other images to have the page format displayed to the 
user in a balanced manner. With the system automatically identifying the image with 
the largest pre-determined print size in group of chosen images to use, the system 
takes this information in order to figure out a way to normalize the image so that the 
image does not dominate the entire image. This is an example of identifying the largest 
pre-determined print size of an image, packing the image in the layout, but also 
normalizing the image so that it does not dominate the overall layout. The image 
identified is considered to have a pre-determined print size since the source of the 
image determined the size of the image before the image was actually used in the 
current system shown in figure 1 to organize and pack images on a layout; see figs. 5- 
10; paragraph [0050]-[0057]); and 

packing as many digital images of the identified largest pre-determined print size 
as possible comprises repeatedly packing digital images of the identified largest pre- 
determined print size in a given orientation until either another digital image of the 
identified largest pre-determined print size will not fit or no digital image of the identified 
largest pre-determined print size remains in the set (i.e. the invention finds images of 
certain sizes that may fill the trial layout in an efficient manner. This may be a large or 
small sized image. The images chosen to fill the trial layout shown in figures 8-10 are 
either the same size or a different size and the feature of packing as many digital 
images of a certain size as possible in a certain orientation in a trial pack is performed. 
The images are placed on the page layout until the images will not fit on the page in 
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order to create a spatial balance. Also, because of the selection criteria listed in 
paragraph [0050], certain images do not remain since the images do not meet criteria 
set by the user. The other feature of no digital image of the identified size remains in 
the set is performed since the identified size may be associated with a time and date 
and if the image of the above time and date criteria does not remain, the above feature 
is performed. With the system automatically identifying the image with the largest pre- 
determined print size in group of chosen images to use, the system takes this 
information in order to figure out a way to normalize the image so that the image does 
not dominate the entire image. This is an example of identifying the largest pre- 
determined print size of an image, packing the image in the layout, but also normalizing 
the image so that it does not dominate the overall layout. With the images identified, 
one, a couple, or all of the images may be considered as the largest in print size if these 
images are compared to the other images used in making up the layout. The images 
identified are considered to have a pre-determined print size since the source of the 
images determined the sizes of the images before the images were actually used in the 
current system shown in figure 1 to organize and pack images on a layout; see figs. 5- 
10; paragraphs [0049]-[0055] and [0059]). 

Re claim 6: Simon et al discloses a method for generating trial packs from a set of 
digital images, each digital image in the set having a pre-determined print size (i.e. 
shown in the illustrated prior art image in figure 2 is a template with the identification of 
a pre-determined print size for each of the digital images for placement on the template. 
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Since this template is used to identify pre-determined print sizes of the pictures, this 
performs the above feature. Also, when the system employs the method of 
automatically placing pictures in the layout, the system chooses, or identifies, a certain 
pre-determined size on the overall layout for placement on the layout to be printed. 
With this pre-determined space, the images chosen have a pre-determined print size 
and this size is taken into account when placing the images on the overall layout. The 
print size of the images is used to assist the system in arranging the images in a more 
aesthetic manner. The images have a pre-determined print size since these images 
where taken by different sources (i.e. digital camera or scanner) and these different 
sources define the print size for the images. Therefore, the pre-determined print size 
feature is performed. The system may then normalize the pictures to take away the left 
over white space on the overall layout when these images are placed on the layout; see 
figs. 1-3; paragraphs [0046]-[0055]), the method comprising 

opening a trial pack as an empty page (i.e. figure 6 is an example of an empty 
trial layout. This shows a view of the page in which the pictures of figure 3 will be 
placed; see fig. 3 and 6; paragraphs [0038] and [0050]); 

continuing, if possible, each open trial pack (i.e. the selection of the images 
placed on the trial layout are automatically selected based on the width, height or 
aspect ratio of the page. With the images chosen automatically, the images are 
continually placed on the page to fit the format chosen and normalizing of the images 
takes place. Normalizing ensures the images are distanced from one another in a 
manner to create a spatial balance between the pictures; see fig. 5; paragraphs [0049]- 
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[0055]) and closing each trial pack that cannot be continued (i.e. once normalizing takes 
place, the images are placed to create a spatial balance. Once this is shown to the user 
for acceptance, the user has a choice to accept the layout or go through the page layout 
process again until an acceptable page layout is obtained. The layout process is 
completed or closed once the user is displayed the new layout; see fig. 5; paragraphs 
[0049]-[0055]); and 

repeating the continuing and closing until no trial pack remains open (i.e. when 
performing the process of comparing and accepting a page layout, the process of 
continuing to apply images to certain layouts and closing the process of adding images 
occurs because the layouts that have the appropriate amount of images on a page are 
repeated until a user decides to accept a displayed layout. At the point where the layout 
is displayed occurs at a point in which the trial layout is closed; see fig. 5; paragraphs 
[0049]-[0055]). 

wherein continuing, comprises, upon determining that at least one digital 
image from the set that has yet to be packed in the open trial pack will fit in the 
packing area (i.e. in the system, a given number of images are to be laid out on a 
certain page layout. The system ensures that this given number is met. If this given 
number is not met for the layout, the system makes the determination and then looks for 
the image to be placed on the page layout to fit the given number of images on the page 
layout. This is an example of the continuing function occurring in the system; see figs. 
5-10; paragraphs [0050]-[0064]): 
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identifying a largest pre-determined print size of a digital image remaining in the 
set that will fit in the packing area (i.e. shown in the illustrated prior art image in figure 2 
is a template with the identification of a pre-determined print size for each of the digital 
images to be used on the template. Since this template is used to identify pre- 
determined print sizes of the pictures to be used, this performs the above feature. Also, 
when the system employs the method of automatically placing pictures in the layout, the 
system chooses, or identifies, a certain pre-determined size on the overall layout to 
place the picture for printing. With this pre-determined space, the images chosen have 
a pre-determined print size and this size is taken into account when placing the images 
on the overall layout. The print size of the images is used to assist the system in 
arranging the images in a more aesthetic manner. The images in the system have a 
pre-determined print size since these images where taken by different sources (i.e. 
digital camera or scanner) and these different sources define the print size for the 
images. Therefore, the pre-determined print size feature is performed. The system 
may then normalize the pictures to take away the left over white space on the overall 
layout when these images are placed on the layout. With the system able to manually 
or automatically choose images to be placed on a layout, a last image that can fit a 
remaining space on the layout may be chosen. This image may be the largest pre- 
determined print size that is able to fit that particular space in the image. When looking 
at figure 14, the cross-hatched space may be used to fit a certain image and the image 
chosen may be the largest print size available to fit in that area. When the system uses 
the method of automatically placing pictures in the layout, the system chooses, or 
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identifies, a certain pre-determined size on the overall layout to place the picture on the 
layout to be printed. In the system, an image may be identified as being much larger or 
smaller than the other images. The image that is much larger than the other images is 
considered as the image with the largest pre-determined print size. This image is 
identified when it is chosen to be placed on the layout and the smallest dimension of the 
image may be normalized in order to create a spatial balance between itself and other 
images; see figs. 1-3; paragraphs [0046]-[0055] and [0057]-[0067]); 

if it will fit, packing a digital image of the identified largest pre-determined print 
size in a first orientation and continuing the open trial pack as a first child trial pack (i.e. 
the amount of images placed on a page is determined automatically based on the layout 
and the amount of space that allows the images to achieve a spatial balance on the 
specified layout. If the image with a pre-determined print size does not assist in 
creating a spatial balance or there is not enough room to fit an image on the page, that 
image is not chosen automatically to be placed on the page. When the first image with 
a pre-determined print size is chosen, another image with another pre-determined print 
size is chosen to see if the first selected images together will create a spatial balance. If 
the images together create a spatial balance, other images are chosen until the 
presentation of an image will take the overall page layout out of a spatial balance. Then 
a score is created to represent the amount of space left on the page layout. The 
process described above is an example of continuing the trial layout as a child trial 
pack. The images chosen can be have the smallest or largest pre-determined print 
size, but the process of normalization is used to assist in creating the spatial balance in 
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the system so that the image sizes do not become a problem to the overall appearance 
of the layout; see fig. 5, 7 and 11-14; paragraphs [0049]-[0057]); and 

if it will fit, packing a digital image of the identified largest pre-determined print 
size in a second orientation and continuing the trial pack as a second child trial pack 
(i.e. during the process that is illustrated in figure 7, the above process that occurs to 
the first or prior trial layout occurs to the new or the second trial layout. The new trial 
layout shows another orientation of the same images, which have the same pre- 
determined print sizes, being used due to the specifications given by the aspect ratio of 
the page format. The above process of the continuing of the trial layout as a first trial 
pack is the same for the second trial layout. The second trial layout is continued as a 
second child trial pack; see fig. 5 and 7; paragraphs [0049]-[0057] and [0059]-[0063]). 

Re claim 8: Simon et al discloses the method of claim 6, wherein: 

packing the digital image of the identified largest pre-determined print-size in the 
first orientation comprises packing as many digital images of the identified largest pre- 
determined print size as possible in the first orientation and continuing the open trial 
pack as a first child trial pack (i.e. the amount of images placed on a page is determined 
automatically based on the layout and the amount of space that allows the images to 
achieve a spatial balance on the specified layout. If the image does not assist in 
creating a spatial balance or there is not enough room to fit an image on the page, that 
image is not chosen automatically to be placed on the page. When the first image with 
a pre-determined print size is chosen, another image is chosen to see if the image sizes 
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together will create a spatial balance. If the print sizes of the images together create a 
spatial balance, other images are chosen until the presentation of an image will take the 
overall page layout out of a spatial balance. Then a score is created to represent the 
amount of space left on the page layout. The process described above is an example 
of continuing the trial layout as a child trial pack. Regarding the largest pre-determined 
print size being chosen and packed, the images used can be the largest or the smallest 
in regards to the print size on the layout. The images may be the largest for the area 
designated on the layout for the image to be placed as shown in figure 17; see fig. 5, 7 
and 11-14; paragraphs [0049]-[0055] and [0057]-[0067]); and 

packing the digital image of the identified largest pre-determined print-size in the 
second orientation comprises packing as many digital images of the identified largest 
pre-determined print size as possible in the second orientation and continuing the open 
trial pack as a second child trial pack (i.e. during the process that is illustrated in figure 
7, the above process that occurs to the first or prior trial layout occurs to the new or the 
second trial layout. The new trial layout shows another orientation of the same images, 
which has the same print sizes, being used due to the specifications given by the aspect 
ratio of the page format. Also, the above process of the continuing of the trial layout as 
a first trial pack is the same for the second trial layout. The second trial layout is 
continued as a second child trial pack; see fig. 5 and 7; paragraphs [0049]-[0055] and 
[0059]-[0063]). 
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Re claim 9: Simon et al discloses the method of claim 8, wherein packing as many 
digital images of the identified largest pre-determined print size as possible comprises 
repeatedly packing digital images of the identified largest pre-determined print size in a 
given orientation until either another digital image of the identified largest pre- 
determined print size will not fit or no digital image of the identified largest pre- 
determined print size remains in the set (i.e. the invention finds images of certain sizes 
that may fill the trial layout in an efficient manner. This may be a large or small sized 
image. The images chosen to fill the trial layout shown in figures 8-10 are either the 
same size or a different size and the feature of packing as many digital images of a 
certain size as possible in a certain orientation in a trial pack is performed. The images 
with certain print sizes are placed on the page layout until the images will not fit on the 
page in order to create a spatial balance. Also, because of the selection criteria listed in 
paragraph [0050], certain images do not remain since the images do not meet criteria 
set by the user. The other feature of no digital image of the identified size remains in 
the set is performed since the identified size may be associated with a time and date 
and if the image of the above time and date criteria does not remain, the above feature 
is performed. Regarding the largest pre-determined print size being chosen and packed, 
the images used can be the largest or the smallest in regards to the print size on the 
layout. The images may be the largest for the area designated on the layout for the 
image to be placed as shown in figure 17; see figs. 5-10; paragraphs [0049]-[0055] and 
[0059]). 
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Re claim 10: Simon et al discloses the method of claim 6, wherein closing comprises, 
for each open trial pack, closing that pack if no digital image from the set that has yet to 
be packed in the open trial pack will fit in the packing area (i.e. when performing the 
process of comparing and accepting a page layout, the process of continuing to apply 
images to certain layouts and closing the process of adding images occurs because the 
layouts have the appropriate amount of images on a page and are repeated until a user 
decides to accept a displayed layout. The trial layouts are closed when the images 
selected create a spatial balance and any more images added to the layout may disrupt 
the spatial balance in the page format chosen. Therefore, when a page layout cannot 
have any images added to the layout because any page added will not fit in order to 
keep a spatial balance, then the page layout is closed; see figs. 5 and 7; paragraphs 
[0049]-[0055]). 

Re claim 1 1 : Simon et al discloses a method of organizing digital images on a page, 
comprising: 

selecting a set of digital images, each digital image in the set having a pre- 
determined print size (i.e. in figure 5, the step 120 allows the images used to be 
arranged on a page format to be selected manually or automatically. Shown in the 
illustrated prior art image in figure 2 is a template with the identification of a pre- 
determined print size for each of the digital images to be used on the template. Since 
this template is used to identify pre-determined print sizes of the pictures to be used, 
this performs the above feature. When the system uses the method of automatically 
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placing pictures in the layout, the system chooses, or identifies, a certain pre- 
determined size on the overall layout to place the picture on the layout to be printed. 
With this pre-determined space, the images chosen have a pre-determined print size 
and this size is taken into account when placing the images on the overall layout. The 
print size of the images is used to assist the system in arranging the images in a more 
aesthetic manner. The images in the system have a pre-determined print size since 
these images where taken by different sources (i.e. digital camera or scanner) and 
these different sources define the print size for the images. Therefore, the pre- 
determined print size feature is performed. The system may then normalize the pictures 
to take away the left over white space on the overall layout when these images are 
placed on the layout; see fig. 5; paragraphs [0047]-[0057]); 

opening a trial pack as an empty page (i.e. figure 6 is an example of an empty 
trial layout. This shows a view of the page in which the pictures of figure 3 will be 
placed; see fig. 3 and 6; paragraphs [0038] and [0050]); 

continuing, if possible, each open trial pack (i.e. the selection of the images 
placed on the trial layout are automatically selected based on the width, height or 
aspect ratio of the page. With the images chosen automatically, the images are 
continually placed on the page to fit the format chosen and normalizing of the images 
also takes place. The normalizing ensures that the images are distanced from one 
another in a manner to create a spatial balance between the pictures; see fig. 5; 
paragraphs [0049]-[0055]) and closing each trial pack that cannot be continued (i.e. 
once normalizing takes place, the images are placed to create a spatial balance. Once 
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this is shown to the user for acceptance, the user has a choice to accept the layout or 
go through the page layout process again until an acceptable page layout is obtained. 
The layout process is completed or closed once the user is displayed the new layout; 
see fig. 5; paragraphs [0049]-[0055]); and 

repeating the steps of continuing and closing until no trial pack remains open (i.e. 
when performing the process of comparing and accepting a page layout, the process of 
continuing to apply images to certain layouts and closing the process of adding images 
occurs because the layouts have the appropriate amount of images on a page and are 
repeated until a user decides to accept a displayed layout; see fig. 5; paragraphs 
[0049]-[0055]) 

comparing the trial closed packs (i.e. at step 240 in figure 7, the two trial layouts 
are compared to see which layout has a greater score in relation to the cost function or 
white space. These trial packs cannot have any more images placed on them and are 
therefore closed when compared; see figs. 6-10; paragraphs [0057]-[0061]); 

selecting a trial pack based upon the comparison (i.e. based on the comparison 
of the trial layouts and their associated scores, the trial layout with the lowest cost 
function or white space score is chosen; see figs. 6-10; paragraphs [0057]-[0061]); and 

determining if any of the digital images from the set were not used in the selected 
trial pack, and if any digital images are determined to not be used, selecting the unused 
digital images as the set of digital images (i.e. in step 100, the digital images to be 
chosen are recognized to be in a database that stores the pictures. In step 120, images 
that are determined to be on the database that are not packed on the current page 
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layout, are selected to be placed on the page. This process can be performed manually 
or automatically. The images that are chosen in step 120 are images that are not 
placed on the page layout before that step has occurred; see fig. 5; paragraphs [0049]- 
[0055]) and repeating the opening, continuing, closing, comparing, selecting, and 
determining (i.e. when another trial layout is being compared to a previous trial layout, 
the steps of opening a new layout page, continuing to place the same images on the 
pages, closing the trial layout after another arrangement is created, then comparing, 
selecting and determining are performed again or repeated. A trial layout may be 
repeated over several times, as illustrated in figure 5, until a desirable layout is 
displayed to the user; see fig. 5 and 7; paragraphs [0049]-[0055] and [0059]-[0063]); 

wherein continuing comprises defining a packing area and upon determining that 
at least one digital image from the set that has yet to be packed in the open trial pack 
will fit in the packing area (i.e. in the system, a given number of images are to be laid 
out on a certain page layout. The system ensures that this given number is met. If this 
given number is not met for the layout, the system makes the determination and then 
looks for the image to be placed on the page layout to fit the given number of images on 
the page layout. This is an example of the continuing function occurring in the system. 
If a user is dissatisfied with the current layout, the system may redefine new areas to 
pack images and place those image in the new areas for packing while making sure that 
the designated number of images to be placed on the overall layout is still performed; 
see figs. 5-10; paragraphs [0050]-[0064]): 
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identifying a largest pre-determined print size of a digital image remaining in the 
set that will fit in the packing area (i.e. shown in the illustrated prior art image in figure 2 
is a template with the identification of a pre-determined print size for each of the digital 
images to be used on the template. Since this template is used to identify pre- 
determined print sizes of the pictures to be used, this performs the above feature. 
When the system employs the method of automatically placing pictures in the layout, 
the system chooses, or identifies, a certain pre-determined size on the overall layout to 
place the picture on the layout to be printed. With this pre-determined space, the 
images chosen have a pre-determined print size and this size is taken into account 
when placing the images on the overall layout. The print size of the images is used to 
assist the system in arranging the images in a more aesthetic manner. The images in 
the system have a pre-determined print size since these images where taken by 
different sources (i.e. digital camera or scanner) and these different sources define the 
print size for the images and therefore, the pre-determined print size feature is 
performed. The system may then normalize the pictures to take away the left over 
white space on the overall layout when these images are placed on the layout. With the 
system able to manually or automatically choose images to be placed on a layout, a last 
image that can fit a remaining space on the layout may be chosen. This image may be 
the largest pre-determined print size that is able to fit that particular space in the image. 
When looking at figure 14, the cross-hatched space may be used to fit a certain image 
and the image chosen may be the largest print size available to fit in that area. When 
the system uses the method of automatically placing pictures in the layout, the system 
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chooses, or identifies, a certain pre-determined size on the overall layout to place the 
picture on the layout to be printed. In the system, an image may be identified as being 
much larger or smaller than the rest of the images. The image that is much larger than 
the others is considered as the image with the largest pre-determined print size. This 
image is identified when it is chosen to be placed on the layout and the smallest 
dimension of the image may be normalized in order to create a spatial balance between 
itself and other images used in the system; see figs. 1-3; paragraphs [0046]-[0055] and 
[0057]-[0067]); 

if it will fit, packing a digital image of the identified pre-determined print size in a 
first orientation and continuing the open trial pack as a first child trial pack (i.e. the 
amount of images placed on a page is determined automatically based on the layout 
and the amount of space that allows the images to achieve a spatial balance on the 
specified layout. If the image with a pre-determined print size does not assist in 
creating a spatial balance or there is not enough room to fit an image on the page, that 
image is not chosen automatically to be placed on the page. When the first image with 
a pre-determined print size is chosen, another image with another pre-determined print 
size is chosen to see if the first images together will create a spatial balance. If the 
images together create a spatial balance, other images are chosen until the 
presentation of an image will take the overall page layout out of a spatial balance. Then 
a score is created to represent the amount of space left on the page layout. The 
process described above is an example of continuing the trial layout as a child trial 
pack. The images chosen can be have the smallest or largest pre-determined print 
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size, but the process of normalization is used to assist in creating the spatial balance in 
the system so that the image sizes do not become a problem to the overall appearance 
of the layout; see fig. 5, 7 and 11-14; paragraphs [0049]-[0057]); and 

if it will fit, packing a digital image of the identified pre-determined print size in a 
second orientation and continuing the trial pack as a second child trial pack (i.e. during 
the process that is illustrated in figure 7, the above process that occurs to the first or 
prior trial layout occurs to the new or the second trial layout. The new trial layout shows 
another orientation of the same images, which have the same pre-determined print 
sizes, being used due to the specifications given by the aspect ratio of the page format. 
Also, the above process of the continuing of the trial layout as a first trial pack is the 
same for the second trial layout. The second trial layout is continued as a second child 
trial pack; see fig. 5 and 7; paragraphs [0049]-[0057] and [0059]-[0063]). 

Re claim 15: Simon et al discloses the method of claim 1 1 , wherein: 

packing the digital image of the identified largest pre-determined print size in the 
first orientation comprises packing as many digital images of the identified pre- 
determined print size as possible in the first orientation and continuing the open trial 
pack as a first child trial pack (i.e. the amount of images placed on a page is determined 
automatically based on the layout and the amount of space that allows the images to 
achieve a spatial balance on the specified layout. If the image does not assist in 
creating a spatial balance or there is not enough room to fit an image on the page, that 
image is not chosen automatically to be placed on the page. When the first image with 
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a pre-determined print size is chosen, another image is chosen to see if the image sizes 
together will create a spatial balance. If the print sizes of the images together create a 
spatial balance, other images are chosen until the presentation of an image will take the 
overall page layout out of a spatial balance. Then a score is created to represent the 
amount of space left on the page layout. The process described above is an example 
of continuing the trial layout as a child trial pack. Regarding the largest pre-determined 
print size being chosen and packed, the images used can be the largest or the smallest 
in regards to the print size on the layout. The images may be the largest for the area 
designated on the layout for the image to be placed as shown in figure 17; see fig. 5, 7 
and 11-14; paragraphs [0049]-[0055] and [0057]-[0067]); and 

packing the digital image of the identified largest pre-determined print size in the 
second orientation comprises packing as many digital images of the identified pre- 
determined print size as possible in the second orientation and continuing the open trial 
pack as a second child trial pack (i.e. during the process that is illustrated in figure 7, the 
above process that occurs to the first or prior trial layout occurs to the new or the 
second trial layout. The new trial layout shows another orientation of the same images, 
which has the same print sizes, being used due to the specifications given by the aspect 
ratio of the page format. Also, the above process of the continuing of the trial layout as 
a first trial pack is the same for the second trial layout. The second trial layout is 
continued as a second child trial pack; see fig. 5 and 7; paragraphs [0049]-[0055] and 
[0059]-[0063]). 
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Re claim 16: Simon et al discloses the method of claim 15, wherein packing as many 
digital images of the identified pre-determined print size as possible comprises 
repeatedly packing digital images of the identified pre-determined print size in a given 
orientation until either another digital image of the identified pre-determined print size 
will not fit or no digital image of the identified pre-determined print size remains in the 
set (i.e. the invention finds images of certain sizes that may fill the trial layout in an 
efficient manner. This may be a large or small sized image. The images chosen to fill 
the trial layout shown in figures 8-10 are either the same size or a different size and the 
feature of packing as many digital images of a certain size as possible in a certain 
orientation in a trial pack is performed. The images with certain print sizes are placed 
on the page layout until the images will not fit on the page in order to create a spatial 
balance. Also, because of the selection criteria listed in paragraph [0050], certain 
images do not remain since the images do not meet criteria set by the user. The other 
feature of no digital image of the identified size remains in the set is performed since the 
identified size may be associated with a time and date and if the image of the above 
time and date criteria does not remain, the above feature is performed. Regarding the 
largest pre-determined print size being chosen and packed, the images used can be the 
largest or the smallest in regards to the print size on the layout. The images may be the 
largest for the area designated on the layout for the image to be placed as shown in 
figure 17; see figs. 5-10; paragraphs [0049]-[0055] and [0059]). 
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Re claim 17: Simon et al discloses the method of claim 11, wherein closing comprises, 
for each open trial pack, closing that pack if no digital image from the set that has yet to 
be packed in the open trial pack will fit in the packing area (i.e. when performing the 
process of comparing and accepting a page layout, the process of continuing to apply 
images to certain layouts and closing the process of adding images occurs because the 
layouts have the appropriate amount of images on a page and are repeated until a user 
decides to accept a displayed layout. The trial layouts are closed when the images 
selected create a spatial balance and any more images added to the layout may disrupt 
the spatial balance in the page format chosen. Therefore, when a page layout cannot 
have any images added to the layout because any page added will not fit in order to 
keep a spatial balance, then the page layout is closed; see figs. 5 and 7; paragraphs 
[0049]-[0055]). 

Re claim 23: Simon et al discloses a computer readable medium having instructions for: 
identifying a set of digital images (i.e. in figure 5, the step 120 allows the images 
used to be arranged on a page format to be selected manually or automatically; see fig. 
5; paragraphs [0049]-[0055]); 

identifying a pre-determined print size for each of the digital images in the set 
(i.e. shown in the illustrated prior art image in figure 2 is a template with the 
identification of a pre-determined print size for each of the digital images to be used on 
the template. Since this template is used to identify pre-determined print sizes of the 
pictures to be used, this performs the above feature. Also, when the system employs 
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the method of automatically placing pictures in the layout, the system chooses, or 
identifies a certain pre-determined size on the overall layout to place the picture on the 
layout to be printed. With this pre-determined space, the images chosen have a pre- 
determined print size and this size is taken into account when placing the images on the 
overall layout. The print size of the images is used to assist the system in arranging the 
images in a more aesthetic manner. The images in the system have a pre-determined 
print size since these images where taken by different sources (i.e. digital camera or 
scanner) and these different sources define the print size for the images. Therefore, the 
pre-determined print size feature is performed. The system may then normalize the 
pictures to take away the left over white space on the overall layout when these images 
are placed on the layout; see figs. 1-3; paragraphs [0046]-[0055]); 

defining a packing area (i.e. when the format of a page is selected in step 110, 
this is analogous to defining a packing area. In this step, the height and width of the 
page is specified in order to define where and within a format the images are to be 
placed on the page; see fig. 5 and 6; paragraph [0049] and [0050]); 

identifying a largest of the predetermined print sizes (i.e. shown in the illustrated 
prior art image in figure 2 is a template with the identification of a pre-determined print 
size for each of the digital images to be used on the template. Since this template is 
used to identify pre-determined print sizes of the pictures to be used, this performs the 
above feature. When the system employs the method of automatically placing pictures 
in the layout, the system chooses, or identifies, a certain pre-determined size on the 
overall layout to place the picture on the layout to be printed. In the system, an image 
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may be identified as being much larger or smaller than the rest of the images. The 
image that is much larger than the others is considered as the image with the largest 
pre-determined print size. This image is identified when it is chosen to be placed on the 
layout and the smallest dimension of the image may be normalized in order to create a 
spatial balance between itself and other images used in the system; see figs. 1-3; 
paragraphs [0046]-[0055]); 

if it will fit in the packing area, packing a digital image from the set having the 
identified largest pre-determined print size set in a first orientation in the packing area in 
a first trial pack (i.e. the amount of images placed on a page is determined automatically 
based on the layout and the amount of space that allows the images to achieve a 
spatial balance on the specified layout. If the image does not assist in creating a spatial 
balance or there is not enough room to fit an image on the page, the current image is 
not chosen automatically to be placed on the page. The system determines the size of 
an area to place an image and judges whether that area should be packed with the 
image that will not overlap other images. This image has a certain print size that needs 
to be taken into account before placing the image in a certain place with other images. 
With this process being performed, this is an example of identifying an image with a pre- 
determined print size to place on the layout in order for the image to not be overlapping 
other images. Once the image is placed, it can be determined that this image is the 
largest image placed on the layout compared to the rest of the images. Next, the 
normalizing process occurs to make sure that the image that is the largest is normalized 
in some manner to create a better spatial balance in the layout and the amount of white 
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space on the layout is minimized. Also, this process occurs until there can not be any 
more images laid out on the trial layout and a score is calculated for the trial layout; see 
fig. 7 and 11-14; paragraphs [0049]-[0055]); and 

if it will fit in the packing area, packing the digital image from the set having the 
identified largest pre-determined print size in the second orientation in the packing area 
in a second trial pack (i.e. during the process that is illustrated in figure 7, the above 
process that occurs to the first or prior trial layout occurs to the new or the second trial 
layout. The new trial layout shows another orientation of the same images being used 
due to the specifications given by the aspect ratio of the page format. Like the above 
explanation, this same system again determines the size of an area to place an image. 
The system judges whether that area should be packed with the image for another trial 
layout if the user has expressed dissatisfaction or the system compares layouts with 
different scores. In this second layout, again the images with a pre-determined print 
size are placed on the layout and these images are placed in a manner that will provide 
ample space for the image to be placed and not overlap any other images in the layout, 
and a normalizing process may occur to assist in improving the visual aspect of the 
layout. This process again occurs until there can not be any more images laid out on 
the trail layout and a score is calculated for the trial layout; see fig. 7; paragraphs 
[0049]-[0055] and [0057]-[0063]). 

Re claim 24: Simon et al discloses the medium of claim 23, having further instruction 
for: 
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wherein the instructions for packing the digital image in the first orientation 
include instructions for, if the digital image from the set having the identified pre- 
determined print size will fit in the first orientation, packing as many digital images from 
the set having the identified largest pre-determined print size as possible in the packing 
area in the first trial pack (i.e. the invention finds images of certain sizes that may fill the 
trial layout in an efficient manner. This may be a large or small sized image. The 
images chosen to fill the trial layout shown in figures 8-10 are either the same size or a 
different size and the feature of packing as many digital images of a certain size as 
possible in a certain orientation in a trial pack is performed. The images in the system 
that are used have a pre-determined size to be printed on the layout. The images are 
placed on the layout with a pre-determined size and these sizes can be the largest or 
the smallest to be placed and printed on the overall layout. The invention can pack 
large images, compared to other images, on a layout as long as the images do not 
overlap. These images can be packed until no other images can be placed on the 
layout anymore. Then, the process of normalization may occur to make the overall 
layout have a better aesthetic look; see figs. 5-10; paragraphs [0049]-[0057] and 
[0059]); and 

wherein the instructions for packing the digital image in the second orientation 
include instructions for, if the digital image from the set having the identified largest pre- 
determined print size will fit in the second orientation, packing as many digital images 
from the set having the identified largest pre-determined print size as possible in the 
second orientation in the second trial pack (i.e. when comparing which trial layout is the 
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most efficient in figure 7, the same process that occurred to the first or prior trial layout 
also occurs to the new or second trial layout. The second trial layout is another 
orientation of the same images and therefore, the above feature is also performed. 
Since the same process for the first orientation occurs for the second orientation, the 
feature of identifying a set of images to place within a pre-determined print space is also 
performed. Like in the first orientation, the images in the system that are used have a 
pre-determined size to be printed on the layout. The images are placed on the layout 
with a pre-determined size and these sizes can be the largest or the smallest to be 
placed and printed on the overall layout. The invention can pack large images, 
compared to other images, on a layout as long as the images do not overlap. These 
images can be packed until no other images can be placed on the layout anymore. 
Then, the process of normalization may occur to make the overall layout have a better 
aesthetic lo; see fig. 5-10; paragraphs [0049]-[0055] and [0059]-[0063]). 

Re claim 25: Simon et al discloses the medium, wherein the instructions for: 

identifying a largest of the pre-determined print sizes (i.e. shown in the illustrated 
prior art image in figure 2 is a template with the identification of a pre-determined print 
size for each of the digital images to be used on the template. Since this template is 
used to identify pre-determined print sizes of the pictures to be used, this performs the 
above feature. When the system employs the method of automatically placing pictures 
in the layout, the system chooses, or identifies, a certain image with pre-determined 
size to be placed on the overall layout to be printed. The image can be the largest or 
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the smallest image out of the images chosen. This performs the feature of identifying 
an image with the largest pre-determined print size to be placed in the layout and 
placing the image or images on the page. The system may then normalize the pictures 
to take away the left over white space on the overall layout; see figs. 1-3; paragraphs 
[0046]-[0055]), comprises 

instructions for identifying, from a set of digital images, a largest pre-determined 
print size that will fit in the packing area (i.e. although identifying the largest pre- 
determined print size is not specifically stated, it is performed by the device. With 
normalization, the largest sized image is isotropically scaled so that the shortest 
dimension of the image is equivalent to the other images used in the page format. The 
image with a certain size, which is identified, is the image that will create a spatial 
balance in the page format. This image can be the largest or the smallest size. The 
image will still be normalized to fit the image together with the other images to have the 
page format displayed to the user in a balanced manner. With the system automatically 
identifying the image with the largest pre-determined print size in group of chosen 
images to use, the system takes this information in order to figure out a way to 
normalize the image so that the image does not dominate the entire image. This is an 
example of identifying the largest pre-determined print size of an image, packing the 
image in the layout, but also normalizing the image so that it does not dominate the 
overall layout. The image identified is considered to have a pre-determined print size 
since the source of the image determined the size of the image before the image was 
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actually used in the current system shown in figure 1 to organize and pack images on a 
layout; see figs. 5-10; paragraph [0050]-[0057]); and 

packing as many digital images of the identified largest pre-determined print size 
as possible comprises instructions for repeatedly packing digital images of the identified 
largest pre-determined print size in a given orientation until either another digital image 
of the identified largest pre-determined print size will not fit or no digital image of the 
identified largest pre-determined print size remains in the set (i.e. the invention finds 
images of certain sizes that may fill the trial layout in an efficient manner. This may be a 
large or small sized image. The images chosen to fill the trial layout shown in figures 8- 
10 are either the same size or a different size and the feature of packing as many digital 
images of a certain size as possible in a certain orientation in a trial pack is performed. 
The images are placed on the page layout until the images will not fit on the page in 
order to create a spatial balance. Also, because of the selection criteria listed in 
paragraph [0050], certain images do not remain since the images do not meet criteria 
set by the user. The other feature of no digital image of the identified size remains in 
the set is performed since the identified size may be associated with a time and date 
and if the image of the above time and date criteria does not remain, the above feature 
is performed. With the system automatically identifying the image with the largest pre- 
determined print size in group of chosen images to use, the system takes this 
information in order to figure out a way to normalize the image so that the image does 
not dominate the entire image. This is an example of identifying the largest pre- 
determined print size of an image, packing the image in the layout, but also normalizing 
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the image so that it does not dominate the overall layout. With the images identified, 
one, a couple, or all of the images may be considered as the largest in print size if these 
images are compared to the other images used in making up the layout. The images 
identified are considered to have a pre-determined print size since the source of the 
images determined the sizes of the images before the images were actually used in the 
current system shown in figure 1 to organize and pack images on a layout; see figs. 5- 
10; paragraphs [0049]-[0055] and [0059]). 

Re claim 26: Simon et al discloses a computer readable medium having computer 
executable instructions for: 

selecting a set of digital images, each digital image in the set having a pre- 
determined print size (i.e. in figure 5, the step 120 allows the images used to be 
arranged on a page format to be selected manually or automatically. Shown in the 
illustrated prior art image in figure 2 is a template with the identification of a pre- 
determined print size for each of the digital images to be used on the template. Since 
this template is used to identify pre-determined print sizes of the pictures to be utilized, 
this performs the above feature. Also, when the system employs the method of 
automatically placing pictures in the layout, the system chooses, or identifies, a certain 
pre-determined size on the overall layout to place the picture on the layout to be printed. 
With this pre-determined space, the images chosen have a pre-determined print size 
and this size is taken into account when placing the images on the overall layout. The 
print size of the images is utilized to assist the system in arranging the images in a more 
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aesthetic manner. The images in the system have a pre-determined print size since 
these images where taken by different sources (i.e. digital camera or scanner) and 
these different sources define the print size for the images and therefore, the pre- 
determined print size feature is performed. The system may then normalize the pictures 
to take away the left over white space on the overall layout when these images are 
placed on the layout; see fig. 5; paragraphs [0047]-[0057]); 

opening a trial pack as an empty page (i.e. figure 6 is an example of an empty 
trial layout. This shows a view of the page in which the pictures of figure 3 will be 
placed; see fig. 3 and 6; paragraphs [0038] and [0050]); 

continuing, if possible, each open trial pack (i.e. the selection of the images 
placed on the trial layout are automatically selected based on the width, height or 
aspect ratio of the page. With the images chosen automatically, the images are 
continually placed on the page to fit the format chosen and normalizing of the images 
also takes place. The normalizing ensures that the images are distanced from one 
another in a manner to create a spatial balance between the pictures; see fig. 5; 
paragraphs [0049]-[0055]) and closing each trial pack that cannot be continued (i.e. 
once normalizing takes place, the images are placed to create a spatial balance. Once 
this is shown to the user for acceptance, the user has a choice to accept the layout or 
go through the page layout process again until an acceptable page layout is obtained. 
The layout process is completed or closed once the user is displayed the new layout; 
see fig. 5; paragraphs [0049]-[0055]); and 
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repeating the continuing and closing until no trial pack remains open (i.e. when 
performing the process of comparing and accepting a page layout, the process of 
continuing to apply images to certain layouts and closing the process of adding images 
occurs because the layouts have the appropriate amount of images on a page and are 
repeated until a user decides to accept a displayed layout; see fig. 5; paragraphs 
[0049]-[0055]); 

wherein the instructions for continuing include instructions for, upon determining 
that at least one digital image from the set that has yet to be packed in the open trial 
pack will fit in the packing area (i.e. in the system, a given number of images are to be 
laid out on a certain page layout. The system ensures that this given number is met. If 
this given number is not met for the layout, the system makes the determination and 
then looks for the image to be placed on the page layout to fit the given number of 
images on the page layout. This is an example of the continuing function occurring in 
the system. If a user is dissatisfied with the current layout, the system may redefine 
new areas to pack images and place those image in the new areas for packing while 
making sure that the designated number of images to be placed on the overall layout is 
still performed; see figs. 5-10; paragraphs [0050]-[0064]): 

identifying a largest pre-determined print size of a digital image remaining in the 
set that will fit in the packing area (i.e. shown in the illustrated prior art image in figure 2 
is a template with the identification of a pre-determined print size for each of the digital 
images to be used on the template. Since this template is used to identify pre- 
determined print sizes of the pictures to be used, this performs the above feature. Also, 
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when the system employs the method of automatically placing pictures in the layout, the 
system chooses, or identifies, a certain pre-determined size on the overall layout to 
place the picture on the layout to be printed. With this pre-determined space, the 
images chosen have a pre-determined print size and this size is taken into account 
when placing the images on the overall layout. The print size of the images is used to 
assist the system in arranging the images in a more aesthetic manner. The images in 
the system have a pre-determined print size since these images where taken by 
different sources (i.e. digital camera or scanner) and these different sources define the 
print size for the images and therefore, the pre-determined print size feature is 
performed. The system may then normalize the pictures to take away the left over 
white space on the overall layout when these images are placed on the layout. With the 
system able to manually or automatically choose images to be placed on a layout, a last 
image that can fit a remaining space on the layout may be chosen. This image may be 
the largest pre-determined print size that is able to fit that particular space in the image. 
When looking at figure 14, the cross-hatched space may be used to fit a certain image 
and the image chosen may be the largest print size available to fit in that area. When 
the system uses the method of automatically placing pictures in the layout, the system 
chooses, or identifies, a certain pre-determined size on the overall layout to place the 
picture on the layout to be printed. In the system, an image may be identified as being 
much larger or smaller than the rest of the images. The image that is much larger than 
the others is considered as the image with the largest pre-determined print size. This 
image is identified when it is chosen to be placed on the layout and the smallest 
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dimension of the image may be normalized in order to create a spatial balance between 
itself and other images used in the system; see figs. 1-3; paragraphs [0046]-[0055] and 
[0057]-[0067]); 

if it will fit, packing a digital image of the identified largest pre-determined print 
size in a first orientation and continuing the open trial pack as a first child trial pack (i.e. 
the amount of images placed on a page is determined automatically based on the layout 
and the amount of space that allows the images to achieve a spatial balance on the 
specified layout. If the image with a pre-determined print size does not assist in 
creating a spatial balance or there is not enough room to fit an image on the page, that 
image is not chosen automatically to be placed on the page. When the first image with 
a pre-determined print size is chosen, another image with another pre-determined print 
size is chosen to see if the first images together will create a spatial balance. If the 
images together create a spatial balance, other images are chosen until the 
presentation of an image will take the overall page layout out of a spatial balance. Then 
a score is created to represent the amount of space left on the page layout. The 
process described above is an example of continuing the trial layout as a child trial 
pack. The images chosen can be have the smallest or largest pre-determined print 
size, but the process of normalization is used to assist in creating the spatial balance in 
the system so that the image sizes do not become a problem to the overall appearance 
of the layout; see fig. 5, 7 and 11-14; paragraphs [0049]-[0057]); and 

if it will fit, packing a digital image of the identified largest pre-determined print 
size in a second orientation and continuing the trial pack as a second child trial pack 
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(i.e. during the process that is illustrated in figure 7, the above process that occurs to 
the first or prior trial layout occurs to the new or the second trial layout. The new trial 
layout shows another orientation of the same images, which have the same pre- 
determined print sizes, being used due to the specifications given by the aspect ratio of 
the page format. Also, the above process of the continuing of the trial layout as a first 
trial pack is the same for the second trial layout. The second trial layout is continued as 
a second child trial pack; see fig. 5 and 7; paragraphs [0049]-[0057] and [0059]-[0063]). 

Re claim 28: Simon et al discloses the medium, wherein the instructions for: 

packing the digital image of the identified largest pre-determined print size in the 
first orientation include instructions for packing as many digital images of the identified 
largest pre-determined print size as possible in the first orientation and continuing the 
open trial pack as a first child trial pack (i.e. the amount of images placed on a page is 
determined automatically based on the layout and the amount of space that allows the 
images to achieve a spatial balance on the specified layout. If the image does not 
assist in creating a spatial balance or there is not enough room to fit an image on the 
page, that image is not chosen automatically to be placed on the page. When the first 
image with a pre-determined print size is chosen, another image is chosen to see if the 
image sizes together will create a spatial balance. If the print sizes of the images 
together create a spatial balance, other images are chosen until the presentation of an 
image will take the overall page layout out of a spatial balance. Then a score is created 
to represent the amount of space left on the page layout. The process described above 
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is an example of continuing the trial layout as a child trial pack. Regarding the largest 
pre-determined print size being chosen and packed, the images used can be the largest 
or the smallest in regards to the print size on the layout. The images may be the largest 
for the area designated on the layout for the image to be placed as shown in figure 17; 
see fig. 5, 7 and 11-14; paragraphs [0049]-[0055] and [0057]-[0067]); and 

packing the digital image of the identified largest pre-determined print size in the 
second orientation include instructions for packing as many digital images of the 
identified largest pre-determined print size as possible in the second orientation and 
continuing the open trial pack as a second child trial pack (i.e. during the process that is 
illustrated in figure 7, the above process that occurs to the first or prior trial layout 
occurs to the new or the second trial layout. The new trial layout shows another 
orientation of the same images, which has the same print sizes, being used due to the 
specifications given by the aspect ratio of the page format. Also, the above process of 
the continuing of the trial layout as a first trial pack is the same for the second trial 
layout. The second trial layout is continued as a second child trial pack; see fig. 5 and 
7; paragraphs [0049]-[0055] and [0059]-[0063]). 

Re claim 29: Simon et al discloses the medium of claim 28, wherein the instructions for 
packing as many digital images of the identified largest pre-determined print size as 
possible include instructions for repeatedly packing digital images of the identified 
largest pre-determined print size in a given orientation until either another digital image 
of the identified largest pre-determined print size will not fit or no digital image of the 
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identified largest pre-determined print size remains in the set (i.e. the invention finds 
images of certain sizes that may fill the trial layout in an efficient manner. This may be a 
large or small sized image. The images chosen to fill the trial layout shown in figures 8- 
10 are either the same size or a different size and the feature of packing as many digital 
images of a certain size as possible in a certain orientation in a trial pack is performed. 
The images with certain print sizes are placed on the page layout until the images will 
not fit on the page in order to create a spatial balance. Also, because of the selection 
criteria listed in paragraph [0050], certain images do not remain since the images do not 
meet criteria set by the user. The other feature of no digital image of the identified size 
remains in the set is performed since the identified size may be associated with a time 
and date and if the image of the above time and date criteria does not remain, the 
above feature is performed. Regarding the largest pre-determined print size being 
chosen and packed, the images used can be the largest or the smallest in regards to 
the print size on the layout. The images may be the largest for the area designated on 
the layout for the image to be placed as shown in figure 17; see figs. 5-10; paragraphs 
[0049]-[0055] and [0059]). 

Re claim 30: Simon et al discloses the medium of claim 26, wherein the instructions for 
closing include instructions for, for each open trial pack, closing that pack if no digital 
image from the set that has yet to be packed in the open trial pack will fit in the packing 
area (i.e. when performing the process of comparing and accepting a page layout, the 
process of continuing to apply images to certain layouts and closing the process of 
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adding images occurs because the layouts have the appropriate amount of images on a 
page and are repeated until a user decides to accept a displayed layout. The trial 
layouts are closed when the images selected create a spatial balance and any more 
images added to the layout may disrupt the spatial balance in the page format chosen. 
Therefore, when a page layout cannot have any images added to the layout because 
any page added will not fit in order to keep a spatial balance, then the page layout is 
closed; see figs. 5 and 7; paragraphs [0049]-[0055]). 

Re claim 31 : Simon et al discloses a computer readable medium having computer 
executable instructions for 

selecting a set of digital images, each digital image in the set having a pre- 
determined print size (i.e. in figure 5, the step 120 allows the images used to be 
arranged on a page format to be selected manually or automatically. Shown in the 
illustrated prior art image in figure 2 is a template with the identification of a pre- 
determined print size for each of the digital images to be used on the template. Since 
this template is used to identify pre-determined print sizes of the pictures, this performs 
the above feature. Also, when the system employs the method of automatically placing 
pictures in the layout, the system chooses, or identifies a certain pre-determined size on 
the overall layout to place the picture on the layout to be printed. With this pre- 
determined space, the images chosen have a pre-determined print size and this size is 
taken into account when placing the images on the overall layout. The print size of the 
images is used to assist the system in arranging the images in a more aesthetic 
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manner. The images in the system have a pre-determined print size since these 
images where taken by different sources (i.e. digital camera or scanner) and these 
different sources define the print size for the images and therefore, the pre-determined 
print size feature is performed. The system may then normalize the pictures to take 
away the left over white space on the overall layout when these images are placed on 
the layout; see fig. 5; paragraphs [0047]-[0057]); 

opening a trial pack as an empty page (i.e. figure 6 is an example of an empty 
trial layout. This shows a view of the page in which the pictures of figure 3 will be 
placed; see fig. 3 and 6; paragraphs [0038] and [0050]); 

continuing, if possible, each open trial pack (i.e. the selection of the images 
placed on the trial layout are automatically selected based on the width, height or 
aspect ratio of the page. With the images chosen automatically, the images are 
continually placed on the page to fit the format chosen and normalizing of the images 
also takes place. The normalizing ensures that the images are distanced from one 
another in a manner to create a spatial balance between the pictures; see fig. 5; 
paragraphs [0049]-[0055]) and closing each trial pack that cannot be continued (i.e. 
once normalizing takes place, the images are placed to create a spatial balance. Once 
this is shown to the user for acceptance, the user has a choice to accept the layout or 
go through the page layout process again until an acceptable page layout is obtained. 
The layout process is completed or closed once the user is displayed the new layout; 
see fig. 5; paragraphs [0049]-[0055]); and 
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repeating the steps of continuing and closing until no trial pack remains open (i.e. 
when performing the process of comparing and accepting a page layout, the process of 
continuing to apply images to certain layouts and closing the process of adding images 
occurs because the layouts have the appropriate amount of images on a page and are 
repeated until a user decides to accept a displayed layout; see fig. 5; paragraphs 
[0049]-[0055]); 

comparing the closed trial packs (i.e. at step 240 in figure 7, the two trial layouts 
are compared to see which layout has a greater score in relation to the cost function or 
white space; see figs. 6-10; paragraphs [0057]-[0061]); 

selecting a trial pack based upon the comparison (i.e. based on the comparison 
of the trial layouts and their associated scores, the trial layout with the lowest cost 
function or white space score is chosen; see figs. 6-10; paragraphs [0057]-[0061]); and 

determining if any of the digital images from the set were not used in the selected 
trial pack, and if any digital images are determined to not be used, selecting the unused 
digital images as the set of digital images (i.e. in step 100, the digital images to be 
chosen are recognized to be in a database that stores the pictures. In step 120, images 
that are determined to be on the database that are not packed on a page layout, are 
selected to be placed on a page. This process can be performed manually or 
automatically. The images that are chosen in step 120 are images that are not placed 
on the page layout before that step has occurred; see fig. 5; paragraphs [0049]-[0055]) 
and repeating the opening, continuing, closing, comparing, selecting, and determining 
(i.e. when another trial layout is being compared to a previous trial layout, the steps of 
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opening a new layout page, continuing to place the same images on the pages, closing 
the trial layout after another arrangement is created, comparing, selecting and 
determining are performed again or repeated. A trial layout may be repeated over 
several times, as illustrated in figure 5, until a desirable layout is displayed to the user; 
see fig. 5 and 7; paragraphs [0049]-[0055] and [0059]-[0063]); 

wherein the instructions for continuing include instructions for defining a packing 
area and upon determining that at least one digital image from the set that has yet to be 
packed in the open trial pack will fit in the packing area (i.e. in the system, a given 
number of images are to be laid out on a certain page layout. The system ensures that 
this given number is met. If this given number is not met for the layout, the system 
makes the determination and then looks for the image to be placed on the page layout 
to fit the given number of images on the page layout. This is an example of the 
continuing function occurring in the system. If a user is dissatisfied with the current 
layout, the system may redefine new areas to pack images and place those image in 
the new areas for packing while making sure that the designated number of images to 
be placed on the overall layout is still performed; see figs. 5-10; paragraphs [0050]- 
[0064]): 

identifying a largest pre-determined print size of a digital image remaining in the 
set that will fit in the packing area (i.e. shown in the illustrated prior art image in figure 2 
is a template with the identification of a pre-determined print size for each of the digital 
images to be used on the template. Since this template is used to identify pre- 
determined print sizes of the pictures to be used, this performs the above feature. Also, 
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when the system uses the method of automatically placing pictures in the layout, the 
system chooses, or identifies, a certain pre-determined size on the overall layout to 
place the picture on the layout to be printed. With this pre-determined space, the 
images chosen have a pre-determined print size and this size is taken into account 
when placing the images on the overall layout. The print size of the images is used to 
assist the system in arranging the images in a more aesthetic manner. The images in 
the system have a pre-determined print size since these images where taken by 
different sources (i.e. digital camera or scanner) and these different sources defined the 
print size for the images and therefore, the pre-determined print size feature is 
performed. The system may then normalize the pictures to take away the left over 
white space on the overall layout when these images are placed on the layout. With the 
system able to manually or automatically choose images to be placed on a layout, a last 
image that can fit a remaining space on the layout may be chosen. This image may be 
the largest pre-determined print size that is able to fit that particular space in the image. 
When looking at figure 14, the cross-hatched space may be used to fit a certain image 
and the image chosen may be the largest print size available to fit in that area. When 
the system employs the method of automatically placing pictures in the layout, the 
system chooses, or identifies, a certain pre-determined size on the overall layout to 
place the picture on the layout to be printed. In the system, an image may be identified 
as being much larger or smaller than the rest of the images. The image that is much 
larger than the others is considered as the image with the largest pre-determined print 
size. This image is identified when it is chosen to be placed on the layout and the 
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smallest dimension of the image may be normalized in order to create a spatial balance 
between itself and other images used in the system; see figs. 1-3; paragraphs [0046]- 
[0055] and [0057]-[0067]); 

if it will fit, packing a digital image of the identified pre-determined print size in a 
first orientation and continuing the open trial pack as a first child trial pack (i.e. the 
amount of images placed on a page is determined automatically based on the layout 
and the amount of space that allows the images to achieve a spatial balance on the 
specified layout. If the image with a pre-determined print size does not assist in 
creating a spatial balance or there is not enough room to fit an image on the page, that 
image is not chosen automatically to be placed on the page. When the first image with 
a pre-determined print size is chosen, another image with another pre-determined print 
size is chosen to see if the first images together will create a spatial balance. If the 
images together create a spatial balance, other images are chosen until the 
presentation of an image will take the overall page layout out of a spatial balance. Then 
a score is created to represent the amount of space left on the page layout. The 
process described above is an example of continuing the trial layout as a child trial 
pack. The images chosen can be have the smallest or largest pre-determined print 
size, but the process of normalization is used to assist in creating the spatial balance in 
the system so that the image sizes do not become a problem to the overall appearance 
of the layout; see fig. 5, 7 and 11-14; paragraphs [0049]-[0057]); and 

if it will fit, packing a digital image of the identified pre-determined print size in a 
second orientation and continuing the trial pack as a second child trial pack (i.e. during 
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the process that is illustrated in figure 7, the above process that occurs to the first or 
prior trial layout occurs to the new or the second trial layout. The new trial layout shows 
another orientation of the same images, which have the same pre-determined print 
sizes, being used due to the specifications given by the aspect ratio of the page format. 
Also, the above process of the continuing of the trial layout as a first trial pack is the 
same for the second trial layout. The second trial layout is continued as a second child 
trial pack; see fig. 5 and 7; paragraphs [0049]-[0057] and [0059]-[0063]). 

Re claim 35: Simon et al discloses the medium, wherein: 

the instructions for packing the digital image of the identified largest pre- 
determined print size in the first orientation include instructions for packing as many 
digital images of the identified pre-determined print size as possible in the first 
orientation and continuing the open trial pack as a first child trial pack (i.e. the amount of 
images placed on a page is determined automatically based on the layout and the 
amount of space that allows the images to achieve a spatial balance on the specified 
layout. If the image does not assist in creating a spatial balance or there is not enough 
room to fit an image on the page, that image is not chosen automatically to be placed 
on the page. When the first image with a pre-determined print size is chosen, another 
image is chosen to see if the image sizes together will create a spatial balance. If the 
print sizes of the images together create a spatial balance, other images are chosen 
until the presentation of an image will take the overall page layout out of a spatial 
balance. Then a score is created to represent the amount of space left on the page 
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layout. The process described above is an example of continuing the trial layout as a 
child trial pack. Regarding the largest pre-determined print size being chosen and 
packed, the images used can be the largest or the smallest in regards to the print size 
on the layout. The images may be the largest for the area designated on the layout for 
the image to be placed as shown in figure 1 7; see fig. 5, 7 and 11-14; paragraphs 
[0049]-[0055] and [0057]-[0067]); and 

the instructions for packing the digital image of the identified largest pre- 
determined print size in the second orientation include instructions for packing as many 
digital images of the identified pre-determined print size as possible in the second 
orientation and continuing the open trial pack as a second child trial pack (i.e. during the 
process that is illustrated in figure 7, the above process that occurs to the first or prior 
trial layout occurs to the new or the second trial layout. The new trial layout shows 
another orientation of the same images, which has the same print sizes, being used due 
to the specifications given by the aspect ratio of the page format. Also, the above 
process of the continuing of the trial layout as a first trial pack is the same for the 
second trial layout. The second trial layout is continued as a second child trial pack; 
see fig. 5 and 7; paragraphs [0049]-[0055] and [0059]-[0063]). 

Re claim 36: Simon et al discloses the medium, wherein the instructions for packing as 
many digital images of the identified pre-determined print size as possible include 
instructions for repeatedly packing digital images of the identified pre-determined print 
size in a given orientation until either another digital image of the identified pre- 
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determined print size will not fit or no digital image of the identified pre-determined print 
size remains in the set (i.e. the invention finds images of certain sizes that may fill the 
trial layout in an efficient manner. This may be a large or small sized image. The 
images chosen to fill the trial layout shown in figures 8-10 are either the same size or a 
different size and the feature of packing as many digital images of a certain size as 
possible in a certain orientation in a trial pack is performed. The images with certain 
print sizes are placed on the page layout until the images will not fit on the page in order 
to create a spatial balance. Also, because of the selection criteria listed in paragraph 
[0050], certain images do not remain since the images do not meet criteria set by the 
user. The other feature of no digital image of the identified size remains in the set is 
performed since the identified size may be associated with a time and date and if the 
image of the above time and date criteria does not remain, the above feature is 
performed. Regarding the largest pre-determined print size being chosen and packed, 
the images used can be the largest or the smallest in regards to the print size on the 
layout. The images may be the largest for the area designated on the layout for the 
image to be placed as shown in figure 17; see figs. 5-10; paragraphs [0049]-[0055] and 
[0059]). 

Re claim 37: Simon et al discloses the medium of claim 31 , wherein the instructions for 
closing include instructions for, for each open trial pack, closing that pack if no digital 
image from the set that has yet to be packed in the open trial pack will fit in the packing 
area (i.e. when performing the process of comparing and accepting a page layout, the 
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process of continuing to apply images to certain layouts and closing the process of 
adding images occurs because the layouts have the appropriate amount of images on a 
page and are repeated until a user decides to accept a displayed layout. The trial 
layouts are closed when the images selected create a spatial balance and any more 
images added to the layout may disrupt the spatial balance in the page format chosen. 
Therefore, when a page layout cannot have any images added to the layout because 
any page added will not fit in order to keep a spatial balance, then the page layout is 
closed; see figs. 5 and 7; paragraphs [0049]-[0055]). 

Re claim 43: Simon et al discloses a method of organizing digital images on a page, 
comprising 

a trial pack generator (i.e. a trial layout is first determined when a layout of 
pictures is generated. Then another score layout is generated due to the 
rearrangement of the same photos in the previous layout. The cost function or white 
space scores are used to compare the two trial layouts and the scores are in relation to 
how much white space is left on the overall layout. The trial layout is considered as the 
trial pack. Although a trial pack generator is not specifically stated, the feature is 
performed; see figs. 6-10; paragraphs [0051]-[0061]); 

and a pack selector (i.e. in the system, depending on the score of the layout, a 
user or the system can selector a trial pack with the image packed in the trial pack. This 
performs the feature of a pack selector; see figs. 5-10; paragraphs [0049]-[0064]) 
wherein: 
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the pack generator is operable to: 

define a packing areas (i.e. when the format of a page is selected in step 110, 
this is analogous to defining a packing area. In this step, the height and width of the 
page is specified in order to define where and within a format the images are to be 
placed on the page. Once an image is placed, the system has to determine which 
section in the image area should be used to place the next image. In the prior art 
section mentioned, figure 2 is described to define packing areas; see figs. 2, 5 and 6; 
paragraph [0049] and [0050]); 

to open a trial pack as an empty page (i.e. figure 6 is an example of an empty 
trial layout. This shows a view of the page in which the pictures of figure 3 will be 
placed; see fig. 3 and 6; paragraphs [0038] and [0050]); 

using a defined packing areas, to repeatedly continue, if possible, each open 
trial pack (i.e. the selection of the images placed on the trial layout are automatically 
selected based on the width, height or aspect ratio of the page. With the images 
chosen automatically, the images are continually placed on the page to fit the format 
chosen and normalizing of the images also takes place. The normalizing ensures that 
the images are distanced from one another in a manner to create a spatial balance 
between the pictures; see fig. 5; paragraphs [0049]-[0055]) and to close each open trial 
pack that cannot be continued until no trial pack remains open (i.e. once normalizing 
takes place, the images are placed to create a spatial balance. Once this is shown to 
the user for acceptance, the user has a choice to accept the layout or go through the 
page layout process again until an acceptable page layout is obtained. The layout 



Application/Control Number: 10/620,505 Page 55 

Art Unit: 2625 

process is completed or closed once the user is displayed the new layout. When 
comparing the trial packs, the trial pack that is being arranged is opened until is it closed 
to be compared to the prior trial pack. This is an example of closing open trial packs 
that cannot have any more images added or rearranged so that the closed trial packs 
can be compared; see fig. 5; paragraphs [0049]-[0055]); and 

wherein the pack generator is operable to continue each open trial pack (i.e. in 
the system, a given number of images are to be laid out on a certain page layout. The 
system ensures that this given number is met. If this given number is not met for the 
layout, the system makes the determination and then looks for the image to be placed 
on the page layout to fit the given number of images on the page layout. This is an 
example of the continuing function occurring in the system. If a user is dissatisfied with 
the current layout, the system may redefine new areas to pack images and place those 
image in the new areas for packing while making sure that the designated number of 
images to be placed on the overall layout is still performed; see figs. 5-10; paragraphs 
[0050]-[0064]) by 

identifying from the set a largest pre-determined print size of a digital image 
remaining in the set that will fit in the packing area (i.e. shown in the illustrated prior art 
image in figure 2 is a template with the identification of a pre-determined print size for 
each of the digital images to be used on the template. Since this template is used to 
identify pre-determined print sizes of the pictures to be used, this performs the above 
feature. Also, when the system employs the method of automatically placing pictures in 
the layout, the system chooses, or identifies, a certain pre-determined size on the 
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overall layout to place the picture on the layout to be printed. With this pre-determined 
space, the images chosen have a pre-determined print size and this size is taken into 
account when placing the images on the overall layout. The print size of the images is 
used to assist the system in arranging the images in a more aesthetic manner. The 
images in the system have a pre-determined print size since these images where taken 
by different sources (i.e. digital camera or scanner) and these different sources define 
the print size for the images and therefore, the pre-determined print size feature is 
performed. The system may then normalize the pictures to take away the left over 
white space on the overall layout when these images are placed on the layout. With the 
system able to manually or automatically choose images to be placed on a layout, a last 
image that can fit a remaining space on the layout may be chosen. This image may be 
the largest pre-determined print size that is able to fit that particular space in the image. 
When looking at figure 14, the cross-hatched space may be used to fit a certain image 
and the image chosen may be the largest print size available to fit in that area. When 
the system employs the method of automatically placing pictures in the layout, the 
system chooses, or identifies, a certain pre-determined size on the overall layout to 
place the picture on the layout to be printed. In the system, an image may be identified 
as being much larger or smaller than the rest of the images. The image that is much 
larger than the others is considered as the image with the largest pre-determined print 
size. This image is identified when it is chosen to be placed on the layout and the 
smallest dimension of the image may be normalized in order to create a spatial balance 
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between itself and other images used in the system; see figs. 1-3; paragraphs [0046]- 
[0055] and [0057]-[0067]); 

if it will fit, packing a digital image of the identified largest pre-determined print 
size in a first orientation in a packing area and continuing the open trial pack as a first 
child trial pack (i.e. the amount of images placed on a page is determined automatically 
based on the layout and the amount of space that allows the images to achieve a 
spatial balance on the specified layout. If the image with a pre-determined print size 
does not assist in creating a spatial balance or there is not enough room to fit an image 
on the page, that image is not chosen automatically to be placed on the page. When 
the first image with a pre-determined print size is chosen, another image with another 
pre-determined print size is chosen to see if the first images together will create a 
spatial balance. If the images together create a spatial balance, other images are 
chosen until the presentation of an image will take the overall page layout out of a 
spatial balance. Then a score is created to represent the amount of space left on the 
page layout. The process described above is an example of continuing the trial layout 
as a child trial pack. The images chosen can be have the smallest or largest pre- 
determined print size, but the process of normalization is used to assist in creating the 
spatial balance in the system so that the image sizes do not become a problem to the 
overall appearance of the layout; see fig. 5, 7 and 11-14; paragraphs [0049]-[0057]); 
and 

if it will fit, packing a digital image of the identified largest pre-determined print 
size in a second orientation and continuing the trial pack as a second child trial pack 
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(i.e. during the process that is illustrated in figure 7, the above process that occurs to 
the first or prior trial layout occurs to the new or the second trial layout. The new trial 
layout shows another orientation of the same images, which have the same pre- 
determined print sizes, being used due to the specifications given by the aspect ratio of 
the page format. Also, the above process of the continuing of the trial layout as a first 
trial pack is the same for the second trial layout. The second trial layout is continued as 
a second child trial pack; see fig. 5 and 7; paragraphs [0049]-[0057] and [0059]-[0063]); 

the pack selector is operable to compare closed trial packs generated by the 
pack generator, to select a trial pack based upon the comparison (i.e. based on the 
comparison of the trial layouts and their associated scores, the trial layout with the 
lowest cost function or white space score is chosen automatically. Although a pack 
selector is not specifically stated, the feature is performed. Also, with the layout having 
the ability to be chosen manually, the feature of a pack selector is believed to be 
performed by the manual function of the invention; see figs. 6-10; paragraphs [0057]- 
[0061]); and, 

until all digital images from the set are used in one of one or more selected trial 
packs, to direct the pack generator to generate new trial packs for any digital images not 
used in a selected trial pack (i.e. in step 100, the digital images to be chosen are 
recognized to be in a database that stores the pictures. In step 120, images that are 
determined to be on the database that are not packed on a page layout, are selected to 
be placed on a page. This process can be performed manually or automatically. The 
images that are chosen in step 120 are images that are not placed on the page layout 



Application/Control Number: 10/620,505 Page 59 

Art Unit: 2625 

before that step has occurred. Also in figure 7, other trial layouts are generated and 
images that are not in the presently generated layout are then placed on that layout to 
compare the current layout to a previous layout; see fig. 5 and 7; paragraphs [0049]- 
[0063]). 

Re claim 46: Simon et al discloses the system of claim 43, wherein the packager is 
operable to, for each open trial pack, close that trial pack if no digital image from the set 
that has yet to be packed in the open trial pack will fit in the packing area (i.e. when 
performing the process of comparing and accepting a page layout, the process of 
continuing to apply images to certain layouts and closing the process of adding images 
occurs because the layouts have the appropriate amount of images on a page and are 
repeated until a user decides to accept a displayed layout. The trial layouts are closed 
when the images selected create a spatial balance and any more images added to the 
layout may disrupt the spatial balance in the page format chosen. Therefore, when a 
page layout cannot have any images added to the layout because any page added will 
not fit in order to keep a spatial balance, then the page layout is closed; see figs. 5 and 
7; paragraphs [0049]-[0055]). 

Claim Rejections - 35 USC § 103 

5. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
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invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

6. Claims 18,19 and 38, 39 are rejected under 35 U.S.C. 1 03(a) as being 
unpatentable over Simon et al in view of Shields (US Pub No 2003/0163786). 

Re claim 18: The teachings of Simon et al are disclosed above. 

However, Simon et al fails to teach the method, wherein defining a packing area 
comprises identifying a geometry of a packed space and defining a packing area 
according the geometry of the packed space. 

However, this is well known in the art as evidenced by Shields. Shields disclose 
defining a packing area comprises identifying a geometry of a packed space (i.e. like 
Simon, Shields discloses rearranging images on a page to mitigate wasted space 
(same field of endeavor). However, the system checks to see if a region of sufficient 
size exists for the placement of an image. Any type of possible shape that is 
represented in figure 2 is recognized to see if the next available image can fit in the 
shape shown; see fig. 2; paragraphs [0012]-[0015]) and defining a packing area 
according the geometry of the packed space (i.e. the area is then designated as an area 
to pack images once an appropriate image of the defined size is found; see fig. 2; 
paragraphs [0012]-[0015]). 

Therefore, in view of Shields, it would have been obvious to one of ordinary skill 
at the time the invention was made to defining a packing area comprises identifying a 
geometry of a packed space and defining a packing area according the geometry of the 
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packed space in order to check to see if a region is a sufficient size for an image to be 
placed (as stated in Shields paragraph [0014]). 

Re claim 19: The teachings of Simon et al are disclosed above. 

Simon et al teaches the method, wherein defining a packing area comprises 
identifying a packed space as rectangular (i.e. when looking at figure 2, it is clear that 
areas that will be packed with an image are or designated as rectangular; see fig. 2; 
paragraph [0048]), identifying left over spaces located diagonally, vertically, and 
horizontally relative to the packed space (i.e. when looking at figure 12, element 62 has 
left over spaces that are horizontal (273) and vertical (275) in relation to the packed 
space. Although identifying spaces diagonally is not specifically stated, it is performed 
by the device. When performing horizontal sorting, the packed space finds the next 
horizontal space and begins to pack that space with an image. If the next image that is 
chosen performs vertical sorting on the now second image process, it will identify the 
diagonal left over space of the first image that was processed horizontally in the 
beginning of whole horizontal process. Therefore, the feature of identifying the diagonal 
left over space is performed; see fig. 12 and 16; paragraphs [0063]-[0067]) and defining 
a second packing area as the remaining horizontal or vertical space (i.e. with horizontal 
and vertical sorting, a packing area or layout area can be defined as a horizontal or 
vertical space when performing the sorting process; see fig. 12 and 16; paragraphs 
[0063]-[0067]). 
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However, Simon et al fails to teach combining the diagonal space with either the 
vertical space or the horizontal space creating a combined space having a maximized 
small dimension, and defining a first packing area as the combined space. 

However, this is well known in the art as evidenced by Shields. Shields discloses 
combining the diagonal space with either the vertical space or the horizontal space 
creating a combined space having a maximized small dimension (i.e. looking at figure 2, 
the space that is left over after placing an image into a certain space is may be 
combined with another space, depending on the image size the system had waiting to 
place on the overall image. Any region of left over space after placing a plurality of 
images can be combined with other pieces of free space left over to create a region to 
fit an appropriate sized image; see fig. 2; paragraphs [0012]-[0015]), and defining a first 
packing area as the combined space (i.e. depending on how the images are oriented, 
space that is left over after the first batch of images are placed is combined to form 
spaces that may be used to have smaller images placed on the combined spaces. In 
the invention, the space is used efficiently by not only placing images in decreasing 
height in the overall image, but also to use the spaces left over to place even smaller 
images. Although it does not specifically say defining a first packing area as a 
combined space, the feature is performed since spaces between the placed images are 
combined to form a region in which images can be placed; see fig. 2; paragraphs 
[0012]-[0015]). 

Therefore, in view of Shields, it would have been obvious to one of ordinary skill 
at the time the invention was made to combining the diagonal space with either the 
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vertical space or the horizontal space creating a combined space having a maximized 
small dimension, and defining a first packing area as the combined space in order to 
check if the next image can be located in one of the openings between the other images 
(as stated in Shields paragraph [0013]). 

Re claim 38: The teachings of Simon et al are disclosed above. 

However, Simon et al fails to teach the method, wherein the instructions for 
defining a packing area include instructions for identifying a geometry of a packed 
space and defining a packing area according the geometry of the packed space. 

However, this is well known in the art as evidenced by Shields. Shields disclose 
the instructions for defining a packing area include instructions for identifying a 
geometry of a packed space (i.e. the system checks to see if a region of sufficient size 
exists for the placement of an image. Any type of possible shape that is represented in 
figure 2 is recognized to see if the next available image can fit in the shape shown. 
Although instructions are not specifically stated, it is clear that the system performs the 
feature of checking for a certain size of a region to place another image. Also, since a 
computer or some program performs the function above, instructions have to be given 
to such a program to perform the above feature; see fig. 2; paragraphs [0012]-[0015]) 
and defining a packing area according the geometry of the packed space (i.e. the area 
is then designated as an area to pack images once an appropriate image of the defined 
size is found; see fig. 2; paragraphs [0012]-[0015]). 



Application/Control Number: 10/620,505 Page 64 

Art Unit: 2625 

Therefore, in view of Shields, it would have been obvious to one of ordinary skill 
at the time the invention was made to have instructions for defining a packing area 
include instructions for identifying a geometry of a packed space and defining a packing 
area according the geometry of the packed space in order to check to see if a region is 
a sufficient size for an image to be placed (as stated in Shields paragraph [0014]). 

Re claim 39: The teachings of Simon et al are disclosed above. 

Simon et al teaches the method, wherein the instructions for defining a packing 
area include instructions for identifying a packed space as rectangular (i.e. when looking 
at figure 2, it is clear that areas that will be packed with an image are or designated as 
rectangular; see fig. 2; paragraph [0048]), identifying left over spaces located 
diagonally, vertically, and horizontally relative to the packed space (i.e. when looking at 
figure 12, element 62 has left over spaces that are horizontal (273) and vertical (275) in 
relation to the packed space. Although identifying spaces diagonally is not specifically 
stated, it is performed by the device. When performing horizontal sorting, the packed 
space finds the next horizontal space and begins to pack that space with an image. If 
the next image that is chosen performs vertical sorting on the now second image 
process, it will identify the diagonal left over space of the first image that was processed 
horizontally in the beginning of whole horizontal process. Therefore, the feature of 
identifying the diagonal left over space is performed; see fig. 12 and 16; paragraphs 
[0063]-[0067]) and defining a second packing area as the remaining horizontal or 
vertical space (i.e. with horizontal and vertical sorting, a packing area or layout area can 
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be defined as a horizontal or vertical space when performing the sorting process; see 
fig. 12 and 16; paragraphs [0063]-[0067]). 

However, Simon et al fails to teach combining the diagonal space with either the 
vertical space or the horizontal space creating a combined space having a maximized 
small dimension, and defining a first packing area as the combined space. 

However, this is well known in the art as evidenced by Shields. Shields discloses 
combining the diagonal space with either the vertical space or the horizontal space 
creating a combined space having a maximized small dimension (i.e. looking at figure 2, 
the space that is left over after placing an image into a certain space is may be 
combined with another space, depending on the image size the system had waiting to 
place on the overall image. Any region of left over space after placing a plurality of 
images can be combined with other pieces of free space left over to create a region to 
fit an appropriate sized image; see fig. 2; paragraphs [0012]-[0015]), and defining a first 
packing area as the combined space (i.e. depending on how the images are oriented, 
space that is left over after the first batch of images are placed is combined to form 
spaces that may be used to have smaller images placed on the combined spaces. In 
the invention, the space is used efficiently by not only placing images in decreasing 
height in the overall image, but also to use the spaces left over to place even smaller 
images. Although it does not specifically say defining a first packing area as a 
combined space, the feature is performed since spaces between the placed images are 
combined to form a region in which images can be placed; see fig. 2; paragraphs 
[0012]-[0015]). 
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Therefore, in view of Shields, it would have been obvious to one of ordinary skill 
at the time the invention was made to combine the diagonal space with either the 
vertical space or the horizontal space creating a combined space having a maximized 
small dimension, and defining a first packing area as the combined space in order to 
check if the next image can be located in one of the openings between the other images 
(as stated in Shields paragraph [0013]). 

7. Claims 20 and 40 are rejected under 35 U.S.C. 1 03(a) as being unpatentable 
over Simon et al in view of Doi et al (US Pat No 6208360). 

Re claim 20: The teachings of Simon et al are disclosed above. 

Simon et al teaches the method, wherein identifying a packing area comprises 
identifying a packed space (i.e. in the layout of the images in figure 5, the area where 
the image will be placed in identified either automatically or manually; see fig. 5; 
paragraphs [0049]-[0055]), maximizing a jagged space (i.e. shown in figures 8-10 are 
examples of the system maximizing the use of spaces that may appeared to be jagged 
or not completely rectangular. Through normalizing and scaling, the jagged spaces in 
figure 8 are maximized by the previous stated methods and is illustrated in figures 9-12; 
see figs. 8-12; paragraphs [0049]-[0055]), identifying remaining spaces that are located 
vertically and horizontally relative to the packed space (i.e. in fig. 16, the blank or white 
spaces located horizontally and vertically of the packed spaces are identified; see fig. 
12 and 16; paragraphs [0063]-[0065]), defining a first packing area as the maximized 
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jagged space (i.e. in figure 14, a crosshatched region is present. This region with the 
surrounding blank region around the crosshatched region is considered as a jagged 
space. The user is utilizing this space in this arrangement, but can multiple different 
arrangements. However, this can be considered as the first packing area as the 
maximized jagged space since it is being used to pack some type of data and the 
crosshatched region is a large region being used in figure 14; see fig. 14; paragraph 
[0067]), defining a second packing area as the left over vertical space (i.e. in using the 
method of figure 16, an area is defined as the vertical white space or the space above 
or below the packed space; see fig. 12 and 16; paragraphs [0063]-[0065]), and defining 
a third packing are as the left over horizontal space (i.e. in using the method of figure 
16, an area is defined as the horizontal white space or the space to the left or right of 
the packed space; see fig. 12 and 16; paragraphs [0063]-[0065]). 

However, Simon et al fails to teach identifying a packed space as irregular. 

However, this is well known in the art as evidenced by Doi '360. Doi '360 
discloses identifying a packed space as irregular (i.e. the Doi system, like the Simon 
system, involves user input to manipulate an overall image in a desired manner (same 
field of endeavor). However, in the Doi system, when viewing a 3D image, the 
irregularity of the shape is recognized by a distance measuring device; see col. 3, lines 
52-58). 

Therefore, in view of Doi '360, it would have been obvious to one of ordinary skill 
at the time the invention was made to identifying a packed space as irregular 



Application/Control Number: 10/620,505 Page 68 

Art Unit: 2625 

incorporated in the device of Simon et al in order to recognize the irregularity of an 
image (as stated in Doi '360 col. 3, lines 52-58). 

Re claim 40: The teachings of Simon et al are disclosed above. 

Simon et al teaches the method, wherein the instructions for defining a packing 
area include instructions for identifying a packed space (i.e. in the layout of the images 
in figure 5, the area where the image will be placed in identified either automatically or 
manually; see fig. 5; paragraphs [0049]-[0055]), maximizing a jagged space (i.e. shown 
in figures 8-10 are examples of the system maximizing the use of spaces that may 
appeared to be jagged or not completely rectangular. Through normalizing and scaling, 
the jagged spaces in figure 8 are maximized by the previous stated methods and is 
illustrated in figures 9-12; see figs. 8-12; paragraphs [0049]-[0055]), identifying 
remaining spaces that are located vertically and horizontally relative to the packed 
space (i.e. in fig. 16, the blank or white spaces located horizontally and vertically of the 
packed spaces are identified; see fig. 12 and 16; paragraphs [0063]-[0065]), defining a 
first packing area as the maximized jagged space (i.e. in figure 14, a crosshatched 
region is present. This region with the surrounding blank region around the 
crosshatched region is considered as a jagged space. The user is utilizing this space in 
this arrangement, but can multiple different arrangements. However, this can be 
considered as the first packing area as the maximized jagged space since it is being 
used to pack some type of data and the crosshatched region is a large region being 
used in figure 14; see fig. 14; paragraph [0067]), defining a second packing area as the 
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left over vertical space (i.e. in using the method of figure 16, an area is defined as the 
vertical white space or the space above or below the packed space; see fig. 12 and 16; 
paragraphs [0063]-[0065]), and defining a third packing are as the left over horizontal 
space (i.e. in using the method of figure 16, an area is defined as the horizontal white 
space or the space to the left or right of the packed space; see fig. 12 and 16; 
paragraphs [0063]-[0065]). 

However, Simon et al fails to teach identifying a packed space as irregular. 

However, this is well known in the art as evidenced by Doi '360. Doi '360 
discloses identifying a packed space as irregular (i.e. when viewing a 3D image, the 
irregularity of the shape is recognized by a distance measuring device; see col. 3, lines 
52-58). 

Therefore, in view of Doi '360, it would have been obvious to one of ordinary skill 
at the time the invention was made to identifying a packed space as irregular 
incorporated in the device of Simon et al in order to recognize the irregularity of an 
image (as stated in Doi '360 col. 3, lines 52-58). 

(10) Response to Argument 

8. Appellant's arguments filed on 1/2/2008 have been fully considered but they are 
not persuasive. 

On page 10 of the brief, the Appellant alleged US Patent Publication of Simon 
does not teach or suggest a method and computer readable medium having instructions 
that include "identifying a pre-determined print size for each of the digital images in the 
set" and "identifying a largest of the pre-determined print sizes". 
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In response to Appellant's arguments that the reference of Simon does not fulfill 
the claim limitations listed above, the Examiner would like to pose a question that gives 
a basis to the Examiner's response. First, within the claim language, what makes the 
print size of each identified digital image pre-determined? With the above question 
posed, the Examiner would like to note that although the claims are interpreted in light 
of the specification, the Examiner cannot read the limitations from the specification into 
the claims. 

The first feature in claims 3, 6, 1 1 , 23, 26 and 31 stating "identifying a pre- 
determined print size for each of the digital images in the set" is taught by Simon in 
several parts of the reference. Given that the claims are interpreted in the broadest and 
reasonable manner, the term "print size" is broadly interpreted as the size of the image 
that will be printed once placed on a page. In the system of Simon '375, it is understood 
that the digital images in the system have a pre-determined printing size. In the system, 
the size of the images placed on the layout can be considered as the pre-determined 
print size since the image will reflect a respective size determined before being placed 
on a page layout when printed. Another example of a pre-determined print size is 
described in the prior art system described within the reference of Simon. In paragraph 
[0004], it is described that the Microsoft Picturelt™ software is used to scale and crop 
images that are to be inserted into album pages. In the prior art system, the user 
defines a size of images by scaling and cropping, and then places the image on the 
album in a manner desired by the user. Here, with the images modified before 
arranging images more efficiently on the album page, the user adjusts the images to a 
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certain size that will be reflected when printed. This is an example of a user performing 
the feature of adjusting the size of the image that will be printed, or identifying a pre- 
determined print size. 

A very basic difference between the prior art system mentioned in Simon and 
Simon's invention is that the invention of Simon performs the same prior art system with 
a program and the program allows the user to choose different layouts of the same 
images that are adjusted for size and arrangement on the page layout. The Simon 
invention is more efficient since it allows for different arrangements of images in a small 
amount of time and is easier to use with more tools at the user's disposal (see the 
scoring of the layout in paragraphs [0051]-[0062]; also see paragraphs [0004] and 
[0005] for prior art reference). 

Both of the prior art system in Simon and Simon's invention read on Appellant's 
claim feature of "identifying a pre-determined print size for each of the digital images in 
the set" since both identify a size of an image that is to be printed and the size of the 
image has been determined before being placed on a page layout in an efficient 
manner. The invention of Simon shows the images that have a size that is already 
determined before being placed on a page to be printed in figure 3 and these images 
may be adjusted through normalization to fit more efficiently on a page layout that is 
shown in figures 8-10 (see paragraphs [0046]-[0063]). Also, the sizes of the images 
may be determined before the printing function. With the size of the images being 
identified (i.e. performing the step of identifying; see figure 5), adjusted through 
normalization and set on the page layout in an efficient manner before printing (i.e. 
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performing the feature of determining print size before printing, or pre-determined print 
size; see figure 5), this is another example of the claim feature identifying a pre- 
determined print size for each of the digital images in the set. 

Because of the broad nature of the claim language, several interpretations were 
expressed on the claim limitations and the Examiner believes that this claim feature is 
performed by the reference. 

Regarding the second feature in claims 3, 6, 1 1 , 23, 26, 31 and 43 stating 
"identifying a largest of the pre-determined print sizes," the Examiner believes this 
feature is also disclosed by Simon. 

In paragraph [0050], Simon clearly states that the purpose of normalization is to 
ensure that one image does not spatially dominate the page layout. In order for this 
situation to occur, one image has to be larger or smaller than the rest of the images on 
the page layout. In order for normalization to occur, the largest image placed on the 
page layout has to be identified in order to know if this image is spatially dominating the 
page layout. With the images reflecting a size to be printed when placed on the page 
layout, or pre-determined print size, the image with the largest height and width is 
identified. The normalization procedure tells the system to find the smallest dimension 
of the largest image and adjust all other images to that smallest dimension (see 
paragraphs [0049]-[0055]). This can occur more than once if the user decides to use 
the system where different scored layouts are compared to one another, as shown in 
figure 7. With the system having to repeat the process of scaling different images and 
normalizing the images in order for the images to fit efficiently on the page layout, the 
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system constantly identifies the largest or smallest images to scale them appropriately. 
Examples of continued scaling of images are shown in figures 9 and 10 (see 
paragraphs [0057]-[0063]). These images are further scaled or normalized in figure 7, 
which is a reflection of step 140 in figure 5, before the printing, storing or transmitting 
operation is performed. Because of the normalization occurring before printing and 
within normalization, the largest size of the image to be printed is identified, the feature 
of identifying the largest of the image size to be printed is performed. The identified 
image size to be printed is equivalent to a pre-determined print size because the image 
size to be printed is determined before the printing operation and since the image size 
to be printed is determined before a printing operation or some other event, this 
performs the above feature of being a pre-determined print size (see fig. 5; paragraphs 
[0050]-[0063]). With the broadest reasonable interpretation of pre-determined print size, 
other interpretations of the Simon reference can be applied to the claim language. 

Lastly, on page 9 and 10, the Appellant makes two remarks that the Examiner 
would like to put on record. On page 9 of the brief, the Appellant states, "The print size 
is not determined until after the images are "normalized." This is in the 4th paragraph of 
the brief on page 9. Also on page 10 the Appellant states in the second paragraph, 
"Furthermore, the largest print size cannot be determined until after the images are 
normalized." See Simon paragraph [0050]." With these two remarks stated in the brief, 
it implies to the Examiner that despite the Appellant's remarks that the Examiner is 
misconstruing the terms "print size" versus "original image size", the Appellant states 
that the teaching of print size is in fact in the Simon reference in paragraph [0050]. 
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Although the Appellant does not go into further detail about this teaching, the Examiner 
would like to point out that the Appellant believes that the print size is, in fact, 
determined in the system of Simon. 

Regarding the 101 rejections, the Examiner would also like to add that the 
specification and claims do not state explicitly that a computer readable medium is only 
comprised of non-transitory subject matter. Due to the recent memo on subject matter 
eligibility of computer readable media, the claimed computer readable medium in claims 
23, 26 and 31 is considered as non-statutory. 

Therefore, in light of the above arguments, the Examiner maintains that the 
reference of Simon does in fact anticipate the present claimed invention for at least the 
reasons stated above. 

(11) Related Proceeding(s) Appendix 

No decision rendered by a court or the Board is identified by the Examiner in the 
Related Appeals and Interferences section of this Examiner's answer. 

For the above reasons, it is believed that the rejections should be sustained. 
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